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New Results on the Existence of Open Loop Nash 
Equilibria in Discrete Time Dynamic Games 

Mathew P. Abraham and Ankur A. Kulkarni 


Abstract —We address the problem of finding conditions which 
guarantee the existence of open-loop Nash equilibria in discrete 
time dynamic games (DTDGs). The classical approach to DTDGs 
involves analyzing the problem using optimal control theory 
which yields results mainly limited to linear-quadratic games 
m. We show the existence of equilibria for a class of DTDGs 
where the cost function of players admits a quasi-potential 
function which leads to new results and, in some cases, a 
generalization of similar results from linear-quadratic games. 
Our results are obtained by introducing a new formulation 
for analysing DTDGs using the concept of a conjectured state 
by the players. In this formulation, the state of the game is 
modelled as dependent on players. Using this formulation we 
show that there is an optimisation problem such that the solution 
of this problem gives an equilibrium of the DTDG. To extend 
the result for more general games, we modify the DTDG with 
an additional constraint of consistency of the conjectured state. 
Any equilibrium of the original game is also an equilibrium of 
this modified game with consistent conjectures. In the modified 
game, we show the existence of equilibria for DTDGs where 
the cost function of players admits a potential fnnction. We end 
with conditions under which an equilibrium of the game with 
consistent conjectures is an e-Nash equilibria of the original game. 

Index Terms —Discrete time dynamic games. Open-loop Nash 
equilibrium. Potential games. Quasi-potential games. Shared 
constraint games. Consistent state conjectures. 

I. Introduction 

Many scenarios which involve more than one player with 
decisions to be made over finitely many stages can be modelled 
as Discrete Time Dynamic Games (DTDGs). In a DTDG, at 
each stage of the game, each player makes decisions which 
optimise his cost function. A set of variables called the state of 
the game evolves according to the state equation depending on 
the decision of players at each stage. Many situations in power 
markets, robotics, network security, environmental economics, 
natural resource economics, industrial organisation and so on 
are known to come under the framework of DTDGs Q, IJI, 
0 . 

The information structure of the dynamic game declares 
what each player knows while making the decision. A com¬ 
monly used information structure is the open-loop information 
structure in which the players only have the information of the 
initial state of the game while making the decisions. This paper 
derives new results on the existence of Nash equilibria under 
the assumption of an open-loop information structure. The 
resulting equilibria are termed as open-loop Nash equilibria. 

An open-loop Nash equilibrium is a profile of strategies 
for which the players do not have any incentive for unilateral 
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deviation. Guaranteeing the existence of an equilibrium has 
been a primary challenge in the theory of DTDGs. In general, 
the approach of analysing DTDGs consists of viewing each 
players’ problem as an optimal control problem and using 
the results from optimal control theory jT]. But the results 
that provide sufficient conditions for the existence of an 
equilibrium are mostly confined to linear-quadratic DTDGs 
0, 0, Q. In this paper, we introduce a new approach of 
analysing DTDGs which leads to new results on the existence 
of open-loop Nash equilibria. Furthermore, in some cases, our 
result generalize earlier results from linear-quadratic games. 

A key step in our approach is the introduction of the 
concept of a conjectured state by players for DTDGs. This 
allows us to model the state of the game as dependent on 
players. This leads to a new formulation of DTDGs with 
the conjectured state also considered as a decision variable. 
In the new formulation, we define a class of games called 
quasi-potential DTDGs. In quasi-potential DTDGs, the cost 
function of players has a special structure. One part of the 
cost function admits a potential function and another part is 
identical for all players. Our first result gives conditions for 
the existence of open-loop Nash equilibria for quasi-potential 
DTDGs. The result is obtained by relating an equilibrium of 
a quasi-potential DTDG to the solution of an optimisation 
problem. 

Following this, we modify the DTDG by introducing an 
additional constraint of consistency of the conjectured statefl 
which can be deciphered as follows. A player in this game 
is constrained to conjecture the state of the game consistently 
with the conjectures of other players. We will refer to this 
game as a DTDG with consistent conjectures. The equilibrium 
that we obtain from the DTDG with consistent conjectures is 
a weaker notion of the equilibrium of the original game. That 
is, an equilibrium of the original game is also an equilibrium 
of the game with consistent conjectures, however, the reverse 
may not be true. The DTDG with consistent conjectures has a 
shared constraint structure unlike the original game. Utilising 
the shared constraint structure, we show the existence of 
equilibria for DTDGs with consistent conjectures under the 
assumption that the cost function of the players admits a 
potential function 13. We end with a result that provides 
sufficient conditions for an equilibrium of the game with 
consistent conjectures to be an e-Nash equilibrium of the 
original game. 

In addition to deriving new results on the existence of open- 
loop Nash equilibria, this approach has other advantages also. 

*The notion of consistent conjecture used here is different from the one 
used by Marie and Tidball in 0 
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For example, the formulation with the conjectured state allows 
one to consider constraints on actions and states in a game. 
Also, the weaker notion of Nash equilibrium for the game 
with consistent conjectures can be used as a design tool since 
there can be some additional equilibrium points in the modified 
game compared to the original one. 

The rest of the paper is organised as follows. The next sec¬ 
tion provides necessary preliminaries and background for the 
paper. Section |III] details the new formulation of DTDGs with 
the conjectured state as decision variable and its equivalence 
to the classical formulation. Section |IV] defines quasi-potential 
DTDGs and conditions for existence of an equilibrium in such 
games. The case of DTDGs with stage-additive cost function 
is also considered for quasi-potential DTDGs in this section. 
Section|V]describes the DTDG with consistency condition, the 
shared constraint structure of this game and the existence of 
an equilibrium in this game with the cost of players having a 
potential function. This section also provides a condition for 
an equilibrium of the DTDG with consistent conjectures to be 
an e-Nash equilibrium of the original problem. The paper ends 
with a conclusion in Section [VI] 

II. Preliminaries & Background 

Before discussing the main contents of the paper, in this 
section, we mention some necessary definitions and results 
which are related to our work. The definition of DTDGs and 
the open-loop Nash equilibria in DTDGs HI are presented 
initially. This will also introduce the notation that we use 
throughout this paper. 

Definition 2.1: A DTDG with open-loop information struc¬ 
ture consists of the following. 

1) An index set, J\f = {1,2,..., N} called the players set, 
where N denote the number of players. 

2) An index set, K. = {1,2,... called the stage set of 
the game, where K is the maximum number of actions 
a player can make in a game. 

3) A set [/{., \/i € JV and Vk G K. called the action /control 
set to which the action of player i at stage k belongs. The 
Cartesian set, U'' = Ul x U^x ■ • ■ x is the action set 
of player i and U = U^x - ■ - x is the action set of the 
game. The action set of adversaries of player i at stage 
k and for the game is defined as * - rijeNMi} 
and C/“* = C/j“* x x • • • x respectively. 

4) A set Xfe, Vfc € 1C\J{K+1} called the state space of the 
game at stage k to which the state of the game belongs. 
Since we are considering the open-loop information 
structure, the initial state of the game is assumed to be 
known to all players and is denoted by xi € Xi. 

5) A mapping fk : Xk x Ul x x ■ ■ ■ x Xk+i 

is defined for each k G 1C such that 

Xk +1 = fk{xk,ul,ul,...,Uk) (1) 

is the state equation of the DTDG, where G Ul,u^ G 
U^, ..., G are the actions of the players at stage 
k and Xk G Xk is the state at stage k. 

6 ) A class of mappings denoted hy T]., i G fif, k G K. 
called the strategy set of player i at stage k. A mapping 


e r{, given by 7 * : JVi is the strategy 

of player i at stage k. The aggregate mapping 7 * = 

K 

{ 7 ?, 72 : • ■ • > 7 ^} G r* = n Tfe is the strategy of player 

fc=i 

i in the game. 

7) A function : U x X 2 x ■ • ■ x Xk+i —K is defined 
for each i G Af called the cost function of Player i in 
the game of K stages. 

For each fixed initial state xi and for fixed A^-tuple of 
permissible strategies { 7 ® G F®; z G A/”}, there exist a 
unique set of vectors {m{, = jKxi);! G N,k G /C} 
and the state evolves according to the state equation O- 
Substitution of these quantities into ,P ,i G Af leads to a 
unique A^-tuple of numbers reflecting corresponding costs to 
the players. This implies the existence of L* : X r2 X 

... X ^ R, the cost function in the strategy space. 

That is L*( 7 \..., 7 ^) = P{u^,... ,u^ ,xi,X2, ■ ■. ,xk+i), 
where 7 * = { 7 ?, 72 , • ■ •, 7if} G T^;i G Af and m* = 
{u\,ul,... ,u]^) G C/®;z G Af and each = Ykixi) and 
Xk+i = fk{xk,ul,ul,.. .,u^),k G /C. 

In a DTDG with open-loop information structure, a player’s 
problem is to decide his strategies 7 ® G F* which minimises 
his cost function U. Often the cost function is expressed as a 
stage-wise additive function which is defined as follows. 

Definition 2.2: In an -person DTDG of AT-stages, player 
z’s cost function is said to be stage-additive if there exist 
functions g\. : ATfe+i x Ul x ■ ■ ■ x x Tffc —>• ]R,Vfc G /C 
such that, 

K 

J\u^,...,u^ ,xi,..., XK+i) = ^ gl[xk+i,ul, ...,Uk, Xk), 

fe=i 

where u® = {u\,... G C/*,Vz G Af and Xk G Xk,'ik G 
/CUjAT-f 1 }. 

We define the open-loop Nash equilibrium in a DTDG as 
follows H]. 

Definition 2.3: An A^-tuple of strategies { 7 ®* G F*;! G A/”} 
constitute an open-loop Nash equilibrium if and only if the 
following inequalities are satisfied Vz G Af. 




(2+1)5*= ^, N ^\ 


y,7^*+i^*,...,7^*),vy gf*. 


Let {zz®* G G Af} be the actions of players correspond¬ 
ing to the equilibrium strategies ( 7 ^*, 7 ^*,... ,'y^*). Unlike 
other information structures, in the open-loop information 
structure, the strategy 7 ^ G F^,, z G Af, fc G /C is a constant 
function of xi. Hence we can write that the TV-tuple of actions 
(zz®* G [/®; z G Af} corresponding to the equilibrium strategies 
constitute an open-loop Nash equilibrium if and only if the 
following inequalities are satisfied Vz G Af. 


J\i 


N* * 

,ZZ ,Xi,X2,. 


‘'if- 1-1 


) < 


zz(®-i)*,zzLu(®+^)*, 




Xi,X2, ■ . .,XK+l),W G U\ 


where x 


k+l — fk{.x}., . 


,..., zz^*), G/C and Xfc+i = 
fk{.Xk,ul*,... ,... ,u^*),k G 1C. 

A commonly used line of analysis for DTDGs is given 
as follows. A necessary condition for the existence of an 
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open-loop Nash equilibrium based on Pontxyagin’s principle 
can be derived as in Ba§ar and Olsder m. A sufficient 
condition for linear-quadratic DTDGs can also be obtained by 
solving a recursive Riccati equation. Under some additional 
assumptions, it is shown in |6l that the two-point boundary 
value problem is uniquely solvable for a linear-quadratic case. 
A recent paper Q by Reddy and Zaccour considers a class 
of linear-quadratic DTDGs with linear constraints on control 
and state. Another potential line of analysis is as follows. 
Since an open-loop Nash equilibrium is equivalently given by 
a sequence of actions m®* G C/®,* S A/" (rather than strategies 
7 ®* G r®,z S TV"), the open-loop Nash equilibrium is also the 
equilibrium of a static game obtained by substituting the state 
equation ([TJ into the cost function of each player. Evidently, 
this line of analysis succeeds only if the cost function and 
the dynamics take simple forms. In this paper, we introduce a 
different approach by reformulating DTDGs with the concept 
of a conjectured state as shown in the next section. 

III. Reformulation of DTDGs 

An important step in deriving the new results on the 
existence of equilibria is the reformulation of DTDGs with 
the notion of the conjectured state. In the classical definition 
of DTDGs, the state of the game is defined as independent 
of players and it evolves “in the background” according to 
the state equation depending on the actions of players. For a 
player, a conceivable manner of playing in a DTDG is to make 
an independent conjecture about the state trajectory. Motivated 
by this, we model the state of the game as dependent on 
players. We will refer to this new formulation as state conjec¬ 
ture formulation. The new formulation has the advantage of 
easier analysis than the classical formulation. Furthermore, in 
this section, we show the equivalence of the state conjecture 
formulation to the classical one. A notion of conjecture is also 
used in |0. But the conjecture used there is about the rivals 
actions using the state or control information which is different 
from our notion of the state conjecture. 

In the state conjecture formulation, the state space of player 
i at stage k is defined as set Vi G G 1C. In order 

to make the new formulation consistent with the original, the 
state space is defined such that Vi G M^Mk G 

1C. The state space of player i over stages is given by X® = 

K+l N 

n Xj. and the state space of the game as X = X®. We 

k=2 i=l 

define the state space of adversaries of player i at stage k as 
X^® = rijeX\{i} space of adversaries of 

_ A -■ 

player i for the game as X ® = X^, ®. The dependence of 

k=2 

the state space on players leads to respective changes in the 
state equation, the strategies and the cost function. The state 
mapping /t : X|, x C7^ x [/| x ... x Xl_^_^ is defined 

for each k G 1C so that, 

= fk{x\,u\,u\, ... ,u^) ( 2 ) 

is the state equation of the DTDG in state conjecture formu¬ 
lation. The permissible strategies in new formulation become 
Yk ■ Xi ^ Ul,i G J\f, k G K. and the cost function of player i 


is a mapping given by J® : [/ x Xi x X 2 x • • • x Xj^^^ —R 
which is defined for each i G Jf. 

1) Player i’s problem: Consider a player’s problem in the 
state conjecture formulation of a DTDG. Fet i G jV be a player 
in the game. Given the rivals actions it“® = («]"*,..., ujf ) G 
t/“® and the initial state xi, the player i’s problem is denoted 
by and is given by the following optimisation 

problem. 


Pfiu ®;xi) 

min J®(it®,x®;M ®,xi) 

s.t. 

(m®,x®) G f2i(M“®;xi), 


where. 


Uj(u"®;xi) = {u\,...,u]^,xI:---Xk+i\uI G Ul,Vk G 1C, 

xl+i G GlC,xl = 

ifc+i = fk{x\,ul,uj:"),yk G 

Since the state trajectory of the game x® = (x\,..., G 

X® is dependent on players, it is also considered as a decision 
variable as opposed to the classical formulation. Define the 
Cartesian product of the feasible set xi) as Cl{u, x) = 

Y\iej\r^i{u~'^-,xi), where u = {v},vf,... ,u^) G U and 
X = {x^,x‘^,... ,x^) G X. The fixed points of the set 
value map Cl{u,x) is denoted by T which is given by 
X = {(u,a;)|(M,x) G U(M,a:)}. We can write the set T in 
the expanded form as follows. 

X = {{u, x)|u G t/, x® G X®, Vi G TV", 0:2 = /i(xi, uj,..., Ui), 
Vi G TV, 4+1 = fk{xl,ul,.. .,u^),\fiG TV,Vfc G /C}. 

Notice that the initial state xi is a parameter on which the set 
X depends. So in the set X, for a fixed xi, the state trajectories 
of players are consistent, i.e., for all i G Af, x® = x^,Wj G 
TV\{i}. Using the state conjecture formulation of DTDG, we 
redefine the open-loop Nash equilibrium as follows. 

Definition 3.1: A tuple of strategies {(u®*,x®*);i G Af} G 
IF is an open-loop Nash equilibrium of DTDG if Vi G Af, 
given M“®* G C/“®, 

J®(w®*,x®*;u-®*,xi) < J®(m®,x®;u-®*,Xi), 

V('«®,x®) G Ui('u“®*;xi). 

2) Equivalence to the classical formulation: The equiva¬ 
lence of the classical and the state conjecture formulation 
in terms of the open-loop Nash equilibria is given in the 
following proposition. 

Proposition 3.1: Consider a DTDG as defined in Definition 
O Suppose the state space is equivalent in the state con¬ 
jecture and the classical formulations (X^i = Xk+i,\li G 
TV, fe G Kf). For a given initial state xi G Xi, then {u*,x*) G 
X is an open-loop Nash equilibrium of a DTDG in the state 
conjecture formulation if and only if it* G t/ is an open-loop 
Nash equilibrium in the classical formulation with the state 
trajectory at equilibrium given by any component x®* G X® 
of X*. 

Proof: Fet (u*,x*) G X be an equilibrium of DTDG in 
state conjecture formulation. Consider the problem of player i 
in state conjecture formulation given by Xi(it“®*; xi), where 
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is the equilibrium actions of other players and Xi, the 
initial state. The player i’s problem in the state conjecture 
formulation with state expanded over stages is denoted by 
problem ; xi) which is given as follows. 


V^{U-^*■,X3) 

min J®(u®,X 2 ,. 

If* 


S.t. 

{o\x\,... 

G H,(m-®*;xi). 


By substituting the state equation (|2|i for the states in the cost 
function from the feasible set, we redefine the problem in 
action space as given below. 



min F{o\ fi{xi,o\,o^"),.. 

If* ,21* 

■ ;o ®*,xi) 

s.t. 

ol G Ul,\/k G 1C. 



This is indeed the classical DTDG problem of player i in the 
action space. Hence, any equilibrium in the state conjecture 
formulation is an equilibrium in the classical formulation. 
Conversely, for any equilibrium in the space of actions, 
player i’s problem can be written as cci) and hence 

an equilibrium in the state conjecture formulation. Hence 
( m * ,x*) G is an equilibrium in state conjecture formulation 
if and only if m* € U is an equilibrium in the original 
formulation. ■ 

Thus, we have a new formulation equivalent to the classical 
one. We will exploit this new formulation to derive our 
existence results. 

IV. QUASI-POTENTIAL DTDG 

Our first result on the existence of open-loop Nash equilibria 
for DTDGs is presented in this section. The result provides 
conditions for the existence of equilibria for a class of games 
called quasi-potential DTDGs. The idea is to split the cost 
function of players into two parts, one part that admits a 
potential function and another part which is identical for all 
players. A quasi-potential game is a special case of a potential 
game ||9l, introduced by Kulkarni et. al. in ifTOl for multi-leader 
multi-follower games. Here, we extend the concept of quasi¬ 
potential to DTDGs. The definition of quasi-potential DTDGs 
is given as follows. 

Definition 4.1: Consider a DTDG defined in state conjec¬ 
ture formulation with the cost function of players given by 
{J^, J^,..., J^}. The DTDG is said to be quasi-potential if 
the cost function of players admits a quasi-potential function 
for which the following has to hold. 

1) There exist functions $i, $ 2 , ■ ■ ■, and h such that 

for all i € fif, player Ts cost function can be written as 
,r{u,xi,x^) = -f h{u,xi,x^),'iu € C/, Vx® € 

X®, 

2) and there exist a function tt such that for all 
i G fif and for all m“® G U“®, we have 

<I’i(u®; m“®, xi) — T>i('u®; u“®, xi) = 7r({(®; it“®, xi) — 
7r(?i®;it“®,Xi),Vu®,'U® G U\ 

The function tt h is termed as a quasi-potential function 
of the quasi-potential DTDG. Note that the function h is 
independent of players, but the argument of the function 


is dependent on players. We utilize this structure of quasi¬ 
potential DTDGs to provide the existence of equilibria in such 
games. 

A. Existence of equilibria in quasi-potential DTDGs 

We need to establish a lemma in order to prove the existence 
of open-loop Nash equilibria in quasi-potential DTDGs. For 
that, consider a set defined as follows. 

. . . ,WK,WK+l)\u\.,K, ■ ■ ■ , 
U^.ji G U,Wk+i G Xfc+i,Vfc G IC,W2 = /i(xi,it},.. 

Wfc-ri = fkiwk,ul,...,u^),yk G 

Note that the initial state xi is a parameter on which the set 
depends. Now we consider the following lemma. 

Lemma 4 . 1 : Consider a DTDG with initial state xi. 
For some i G M, given m“® G C/“®, a point 
{u'^, 1x2,11:3,... ,wk+i) is feasible for the problem given by 
Pi{u~'^; xi) if and only if {u, 102,1x3,..., wk-\-i) G That 
means, 

(vl,1x2,1x3,...,WK+\)GGLi{yr'--,xf) < 5 =^ 

(u, 1X2, 1X3,..., WK-i-l) G 

Proof: For given it“® G C/“® and initial state 

xi, consider a point {u^ , 1 x 2 , 003 ,... ,WKJri) G Hi(u“®;xi). 
That means, u® G U^,W2 = fi{xi,u\,uf^),ixk+i = 
fk{iXk,u\,uf'^),'ik G /C\{1}. Now by combining u® G U® 
and it“® G t/“® we can rewrite the above equations as 
u GU,1x2 = fi{xi,u\,uf''),Wk+i = fk{iXk,u\,uf''),\/k G 
/C\{1}. Hence, {0,002,1x3,... ,ixk+i) G 

“- 4 =” Suppose {u,W 2 ,iX 3 ,... ,wk+i) G J-^. Hence, u G 
U,iX2 = fi{xi,u\,uf''),Wk+i = fk{ixk,ul,uf'-),yk G 
/C\{1}. For some i G N, we separate the actions of player i 
and adversaries as rt® G C/® and m“® G C/“®. For some fixed 
w“® G C/“® and xi, the equations can be rewritten as m® G 
U\W2 = fi{xi,u\,uf'-),iXk+i = fk{iXk,u\,uf'-),\/k G 
/C\{1}. Hence, (m®, 1x2,003,..., wk+i) G xi). ■ 

For a quasi-potential DTDG with quasi-potential function 
TT -\- h, consider an optimisation problem denoted by Pq. 

Pq min tt{u,xi) + h{u,xi,iX 2 ,iX 3 ,... ,wk+i) 

(u,W2,...,WK + l) 

s.t. (0,102,1X3, ... ,1XK-l-l) G 

Note that the objective function of the problem Pq is the quasi¬ 
potential function tt -\- h and the feasible set is F'^. We relate 
the solution of Pq to an equilibrium of a quasi-potential DTDG 
which is given in the following theorem. 

Theorem 4 . 1 : Consider a quasi-potential DTDG with a 
quasi-potential function tt -\- h. Consider the optimisation 
problem given by Pq for the quasi-potential DTDG. If 
{0,1x2,003,... ,iXK+i) G is a global minimizer of the 
problem Pq, then {u, x) G F is an open-loop Nash equilibrium 
of the quasi-potential DTDG with x^._|_]^ = iOk+i , Vfc G /C, Vi G 

M. 

Proof: Let {0,1x2,003,..., ixk+i) G F^ be the minimizer 
of the problem Pq. Then, 

tt{o, Xi) -I- h{u, xi,W2,1X3,..., ooK+i) < ti{u),xi + h{o, Xi, 
W2,. .. ,1XK+l),'^{0,002,1X3,. .. ,OOK+l) G F'^. 
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By splitting the actions u,u € U over player i as m = (m®, u ®) 
and u = respectively, we can rewrite as, 

Tr{u\u~\xi) + h{u\u~\xi,W2,W3,.. .,wk+i) < 
'k{u\u~\xi) + h{u\xi,u~\w2,W3,. . . ,WK+l), 
\/{u\u~\w2,W3, . . .,WK+l) e 

The inequality still holds even if we replace S by 
u“® G [/“*. Hence, the above inequality can be rewritten as, 

Tr{u\u~\xi) + h{u\u~\xi,W2,W3,... ^wk+i) < 

Tr{u\ u~\ xi) + h{u\ u~\ , X1W2, W3,..., wk+i), 
\/{u\u~\w 2 ,W 3 , ■ ■ .,WK+l) € 

Using Lemma ItTI the above inequality can be rewritten as, 

tt{u\u~\xi) + h{u\u~\xi,W2,'W3,... ,wk+i) < 

Tr{u\ u~\ a:i) + h{u\ u~\ xi,W2,W3,..., wk+i), 
\/{u\w 2 ,'W 3 , ■ ■ ■ ,wk+i) € G Af. 

Since the game is a quasi-potential DTDG and 

{W2,W3,.. .,'WK+i) = x® G X\yi G Af, 

u“®, Xi) + /l(u®, M“®, Xi,x'^) < 

u~\ Xi) + h{u\ u~\ xi,W2,W3,---, Wk+i), 
y{u\w2,W3, ■ ■ ■ ,wk+i) G Hi('u“®;xi),Vz G Af. 
Hence, Vi G Af, 

F{u,x\xi) < F{u\xi,W2,W3, ■ ■., Wic+i;u“®), 

V('U®, W2,W3,..., WK+i) G Hi(it“®; Xi). 


Thus, if the optimisation problem Pq has a minimizer then 
there exists an open-loop Nash equilibrium for the quasi¬ 
potential DTDG. The following results gives conditions which 
guarantee the existence of a minimum for the problem Pq. 

Corollary 4.2: A quasi-potential DTDG admits an open- 
loop Nash equilibrium if the quasi-potential function, tt + h 
is continuous and the set F'^ is non empty and compact. The 
set F^ is compact if the sets U, X are compact and fk is 
continuous for all fc G /C. 

Corollary 14.21 can be easily proved using Weierstrass Theo¬ 
rem. But the requirement of compactness of state space in 
Corollary 14.21 may be restrictive in some cases. The following 
result gives the existence of equilibria without the condition 
of compactness of state or action spaces, instead imposing 
the condition of coercivity of quasi-potential function. The 
coercivity definition is given as follows. 

Definition 4.2: A quasi-potential function tt + his coercive 
if 

liminf 7r(u, xi) -|- h{u, xi,W2, ■. ■, wk+i) = 00 . 

I lu,W2,...,WK + l I |— 

(u,W 2 ,...,WK + i)£J^'’ 

Corollary 4.3: A quasi-potential DTDG admits an open- 
loop Nash equilibrium if the quasi-potential function, tt + h is 
continuous and coercive and fk is continuous for each k 1C. 

Since the state conjecture formulation is equivalent to the 
original formulation, an equilibrium of a DTDG in this new 


formulation with cost function of players admitting a quasi¬ 
potential function is indeed an equilibrium in the original 
formulation. 


B. Stage-additive quasi-potential DTDG 

Now we consider the case of stage-additive DTDG as 
defined in Definition 12.21 For such games the stage-wise cost 
functions g^, g^,..., admits a quasi-potential function if 
the following conditions are satisfied for each k & 1C. 

1) There exist functions and hk such that 

Vi G Af, g\ is given by, 

9ki.^\+i,'^\, ■ ■ ■ ,'t'k ,^k) = ■ ■ ■ ,Uk) + hk{xk, 

ul.. .,u^,xU,),\/ul G Ul\/xl+, G X^+i,Vfc G 1C, 

2) and there exist functions TTk,'Ck G /C such that for all 
i C Af and for all itjT® G U^®, 

T^kiul,u-^),\/ul,ul G Ul. 

The function tt^ -\- hk is 3 stage-wise quasi-potential func¬ 
tion for the stage-wise quasi-potential DTDG. The following 
proposition shows that if the stage-wise cost function of 
players admit a stage-wise quasi-potential function, then the 
overall cost functions also admit a quasi-potential function. 

Proposition 4.4: Consider a stage-additive DTDG defined 
in Definition 12.21 If the stage-wise cost functions gl,i € Af 
admit a stage-wise quasi-potential function, then the stage- 
additive cost functions F, F,..., also admit a quasi¬ 
potential function. 

Proof: The stage-additive cost function of player z in a 
stage-wise quasi-potential DTDG is given by. 


J\F,v?,... ,u^ ,xi,x\,. . . , Xk+i) = 

K K 

• ■ •) ) -b ^ hk{xl,ul, ...,U^, x^+i), 

fc=l k=l 


where u\ G Ul,u^ G U^,x\^.^ G 2f^._|_;^,Vz G Af. The first 

K 

sum admits a potential function given by TTk- The second 

fc=i 

sum is independent of players though the argument depends 
on. Hence, the stage-additive cost function of players admit a 
quasi-potential function. ■ 

For a stage-wise quasi-potential DTDG with stage-wise 
quasi-potential function tt^ -b hk, consider the following opti¬ 
misation problem denoted by P® with wi = xi. 


Pq min 

{u,W2,...,WK + l) 

s.t. 


K r 


TTk{ul,...,U^)-\-hk{u,Wk,Wk+l) 

k=l L 

{u,W2,W3, . . .,WK+l) G F^. 


It can be seen from Theorem 14.11 that the minimizer of the 
problem P® is an open-loop Nash equilibrium of the stage- 
wise quasi-potential DTDG with x^._|_]^ = Wk+i,'Ci G Af,'Ck G 
/C. The problem Pq has the structure of a standard control 
problem for which a vast theory is available which can be 
utilized to determine the equilibria. 





6 


It can be seen that the linear-quadratic DTDGs under 
certain assumptions comes under the class of stage-wise quasi¬ 
potential DTDGs. We first provide the definition of linear- 
quadratic DTDGs. 

Definition 4.3: An A^-person DTDG is of linear-quadratic 
type if Ul = ]R™(i eJ\f,kG K.), and 

= Akxl + blul, 


9kixk+n'^k^ • ■ •) '^k tX\) — - + 

jeAf 


where Ak, Bk,Ql., are matrices of appropriate dimen¬ 
sions, is symmetric, >- 0 and u\. S S 

G Ar,Vfc G 1C. 

Lemma 4.2: For a linear-quadratic DTDG with Qk+i = 
Qk+i = Qk+iy^Cj C N.'ik G 1C, the stage-wise cost 
function of players, admit a quasi-potential function. 

Proof: For a linear-quadratic DTDG, the stage-wise cost 
function is given by. 


5fc(a 


fc+1’ ’ 1 ’ 


k) — 2 + 

Y ^k 

few 


Ml 


It can be easily verified that the second summation term 
is a quadratic function which admits a potential func¬ 
tion with stage-wise potential function TTk{u\,... ,u^) = 

Hence, in the definition of quasi-potential 

i=l 

DTDG, 'I'fc = 5 X] '“fe- Also, the first function is 

jeA' 

identical for all players and the argument depends on the state 
conjectured by players. Hence, hk = which 

implies that admits a quasi-potential function. ■ 

The fact that the stage-wise cost function of the linear- 
quadratic DTDGs under certain assumptions admit a stage- 
wise quasi-potential function can be utilized to provide the 
existence of equilibria in such cases which are given by the 
following corollary. 

Corollary 4.5: Consider a linear-quadratic DTDG as de¬ 
fined in 14.31 Suppose the assumptions of Lemma 14.21 hold 
with Qk+i 0,Vfc G 1C. Then there exists an equilibrium for 
the linear-quadratic DTDG which is given by the minimizer 
of the problem P^. 

Proof: Since Qk+i 0,Vfc G 1C and 0,Vi G 

M,\lk G 1C, the quasi-potential function ^ (tt^ -|- hk) as 

_ fee/c 

defined in the proof of Lemma 14.21 for the linear-quadratic 
DTDG is coercive. Then the result follows directly from 
Corollary 14.31 ■ 

Thus, using the state conjecture formulation, the existence 
of open-loop Nash equilibria can be guaranteed for games 
with cost function of players admitting quasi-potential function 
which also includes a class linear-quadratic DTDGs. In the 
next section, we introduce some additional constraints on the 


state conjecture formulation which can provide new existence 
results for DTDGs. 

V. DTDG WITH CONSISTENT CONJECTURES 

In the state conjecture formulation of DTDGs, each player 
conjectures the state of the game independently. In this sec¬ 
tion, we modify the DTDG with an additional constraint of 
consistency of the conjectured states. This constraint requires 
that each player conjecture the state of the game consistently 
with other players. We will refer to this game as a DTDG 
with consistent conjectures. Even though the DTDG with 
consistent conjectures is different from the original DTDG, 
an equilibrium of the original game is also an equilibrium 
of the game with consistent conjectures. The DTDG with 
consistent conjectures has a special structure called shared 
constraint structure which the original game does not have. 
We utilize this structure for giving conditions for the existence 
of equilibria for DTDGs when the cost functions of players 
admit a potential function. In this section, we also discuss 
conditions under which an equilibrium of the DTDG with 
consistent conjectures is an e-Nash equilibrium of the original 
game. 

A player’s problem in the game with consistent conjectures 
is given in the following section. 

1) Player i’s problem in the DTDG with consistent con¬ 
jectures: Let i G A/^ be an arbitrary player in a DTDG with 
consistent conjectures. Given m“® = {uf\ ... ,uff) G 
the actions of other players, X ® = {x2\...,Xjf_^_^) e X \ 
the state conjectures of other players and xi, the initial state 
of the game, player z’s problem in this game is denoted by 
P^ (m“®, xi) which is given by the following. 


H 

T 

T 

'4 

min J®(m®,x®;m ®,xi) 

S.t. 

(m®,x®) G H'(m"®,x“®;xi), 


where. 


H'(u \x ®;a;i) = {(M®,i®)|M®fc G C/^,Vfc GG X^+ 1 , 

V/c G /C,X 2 = fl{xl,u\,uf"),xl_^_l = fk{xl,ul,uf"), 

Vfc G lC\{l},xl+, = G Ar\{*},Vfc G 1C}. 

The Cartesian product of the feasible set xi) over 

i G JV is given by the set value map denoted by Cl (m, x) which 

is given by. Cl {u,x) = f2-(M“®,Xi), where u G U 

i&AT 

and X G X. The fixed points of the set value map Cl (m,x) 
is denoted as P' and is given by, P' = {(m,x)|(m,x) G 
Cl'{u,x)}. It can be given by, 

P = {{u,x)\u G U,x G X,X 2 = /i(xi, M^,... G J\f, 

4+1 = fk{x\,ul,...,u^),'Ci GffykG 1C, 

4+1 = 4+u^* CAf,'Cj Gfif\{i},yk G X}. (3) 

Notice that since the consistency condition is redundant in the 
set P , the set P is same as P. That is, 

P' = P. 
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The definition of open-loop Nash equilibrium for a DTDG 
with consistent conjectures is given as follows. 

Definition 5.1: Consider a DTDG with consistent conjec¬ 
tures with the initial state xi. A point {(u®*, x**); i £ M} £ J-' 
is said to be an open-loop Nash equilibrium of the DTDG with 
consistent conjectures if Vi £ N, given x 

X~'^, the actions and state conjectures at the equilibrium, 

'i[u\x^) £ nfiu-",x-^*]xi). 

2) Relation to equilibria of the original game: The relation 
of equilibria of the original game to equilibria of the game with 
consistent conjectures is given by the following proposition. 

Proposition 5.1: Consider a DTDG in state conjecture 
formulation. If ,... ,u^*,x^*,... ,x^*) £ is an 

open-loop Nash equilibrium of the original DTDG, then 
(u^*,..., u^*,x ^*,..., x^*) £ P is also an open-loop Nash 
equilibrium of the DTDG with consistent conjectures. 

Proof: Since IF = -F, an equilibrium of the orig¬ 
inal DTDG is also feasible for the DTDG with consis¬ 
tent conjectures. For player i £ N, let £ 

[/“* X X”* be the actions and state conjectures of other 
players at the equilibrium of the original game. Since 
the set Xi) has additional constraints than 

ni(u“**;xi), for all * £ A/", x“**;Xi) C 

f2,(u-”;xi). Hence ,... ,u^*,x^*,... ,x^*) £ P is 
also an open-loop Nash equilibrium of the DTDG with con¬ 
sistent conjectures. ■ 

Thus, an equilibrium of the original game is also an equilib¬ 
rium of the DTDG with consistent conjectures. The reverse 
relation will be discussed in Section IV-CI Next we show the 
shared constraint structure of the mapping H which is vital 
for the existence result. 

A. Shared constraint structure of the DTDG with consistent 
conjectures 

The motive of modifying the game with consistent conjec¬ 
tures is that it gives a shared constraint structure to the game 
which can be utilized for deriving an existence result. The 
relation of the consistent conjectures and the shared constraint 
structure applied to multi-leader multi-follower games is found 
in CD. The set value map H (it, x) is a shared constraint 
mapping if there exists a set S such that Vi £ Af, 

(tt\x*) £ x“*; Xi) -4=^ (u,x) £ S. 

Notice that the set S is independent of i. The concept of 
shared constraint structure was studied in 1965 by Rosen ca 
and is used in some recent papers (e.g., C3, d, C3). 
The following lemma shows that for a DTDG with consistent 
conjectures, the set value map fl (it, x) has a shared constraint 
structure. 

Lemma 5.1: Consider a DTDG with consistent conjectures. 
Let H (it, x) be the Cartesian product of the constraint set of 
problem (it“®, x“®; xi). Then VL (it, x) is a shared constraint 
mapping with shared constraint set given by X = {U x AjCXH, 


where 

-4= {(x*,x"*)|(x\x"*) £ X,x ^+1 = x'(,+i,Vj £ A/'\{i}, 

Vfc £ /C}, 

and TL = {(it*, it“*, x*, x“*)|(it*, it“*) £ C/, (x*, x“*) £ X, 

4 = = fk{xl,ul,uf")Pk £ /C\{1}}- 

Proof: Let i £ A/” be an arbitrary player for the game with 
consistent conjectures. By using the defined sets A and PL, the 
set f2j(ii“®,x“Va^i) can be written as, ,x~'‘-,xi) = 

{{u\P)\{u\u-^) &U,{x\x-^) £ A,{u\u-\xfx-^) £ 
Pi}, i.e., (V, X®) £ x“®; Xi) (it, x) € {U x A) Cl 

PL, which is independent of i. Since the implication holds for 
each 1 £ A/", H (it, x) is a shared constraint mapping with 
shared constraint set given by {U x ,4) fl PL. It can be seen 
from (O that the shared constraint set is indeed the set of fixed 
points of the mapping H . Hence, F = {U x A) D PL. ■ 

Thus, we have, 

(ii®,x*) £ f2j(it“*, x“®; xi) (u,x) € F . (4) 

Now we use the relation (HI to show the existence of equilibria 
for a class of games called the potential DTDGs in the next 
section. 

B. Existence of equilibria for a potential DTDG with consis¬ 
tent conjectures 

Potential games were well studied by many and one pivotal 
paper is El by Monderer and Shapley. We extend this concept 
to DTDGs and term it as potential DTDGs which is defined 
as follows. 

Definition 5.2: Consider a DTDG in conjecture state formu¬ 
lation as defined in Section |III] with cost function of players 
given by {J^, P,..., J^}. The DTDG is said to be potential 
if there exist a potential function tt such that for all i £ AT 
and for all ii“® £ U~\ 

J®('u*, X*, it“®; xi) — J®(ii®, X*, it“®; xi) = 7r(i(®,x®,it“*;xi) — 
77 ( 11 *, x®,ii“*;xi),V('u*,x*), {iT‘,F) £ C/* x X®. 

The advantage of a potential game is that the minimize! of the 
potential function El over suitably defined feasible set gives 
an equilibrium of the game. Conditions for the existence of 
open-loop Nash equilibria in a potential DTDGs is given by 
considering the following optimisation problem denoted by P. 

P min 7r(it,x;xi) 

U,X 

s.t. (u, x) £ -F . 

Note that the objective function of problem P is the potential 
function, tt which is to be minimized over the shared constraint 
set F . The following theorem shows that a minimize! of the 
problem P is an open-loop Nash equilibrium of the potential 
DTDG with consistent conjectures. 

Theorem 5.2: Consider a DTDG with consistent conjec¬ 
tures. Let F be the shared constraint set of the DTDG with 
consistent conjectures. Suppose the cost function of players 
admit a potential function tt, then any minimize! of problem 





P is an equilibrium of the potential DTDG with consistent 
conjectures. 

Proof: Suppose {u*,x*) G P is a global minimum of 
the problem P. Then, 

7r(u*,a:*;xi) < 7r(u,cc;cci), 'i{u,x)GP. 

For some i G J\f, split the argument u = G U and 

X = G X, we can rewrite the above inequality as, 

i:[u , X ; xi) < 7r(u ,x,x ,x,x 

The inequality still holds even if we replace and 
by and x“**, the actions and state conjectures of other 
players at the minimum. That is, 

tt{u* ,x*]Xi) < 7r(u*,x*,x“**;xi), 

V(u*,u-**,x*,x-”) G P' . 

Since is a shared constraint mapping of (u, x), by 
Lemma O (u*,u”**,x*,x"”) G P' (ufx^) G 

x“®*;xi). Hence, we can rewrite the inequality as, 

7r('«*,X*;xi) < 7r(it*,x*,x“**;xi), 

VK,x*) gH'(u-**,x-“;xi). 

Since tt is a potential function of the DTDG, Vi G M, 

r{u^*,P*,u-^*,x-^*;xi) < r{u\x\u-^*,x-^*;xi), 
V(it*,x*) G x“**;xi). 

Hence, {u*,x*) G P' is an open-loop Nash equilibrium of the 
potential DTDG with consistent conjectures. ■ 

Thus, for a potential DTDG with consistent conjectures, a 
minimiser of the problem P is an open-loop Nash equilibrium. 
Conditions under which an equilibrium exist for a potential 
DTDG with consistent conjectures is given in the following 
corollaries. 

Corollary 5.3: Consider a potential DTDG with consistent 
conjectures. Suppose the potential function tt is continuous and 
the set P is compact, then the game has an equilibrium given 
by the minimizer of the problem P. The shared constraint set, 
P is compact if the sets U, X are compact and fk, the state 
mapping is continuous for all k G X. 

Conditions for the existence of an equilibrium without the 
compactness of state space and action set is given as follows. 

Corollary 5.4: Consider a potential DTDG with consistent 
conjectures. Suppose the potential function tt is continuous 
and coercive and fk is continuous for all k G X, then the 
game has an equilibrium. 

Thus, for a potential DTDG with consistent conjectures under 
these conditions we can guarantee the existence of an open- 
loop Nash equilibrium and the equilibrium is given by a 
minimizer of problem P. 

C. e-Nash equilibrium of the original game 

We have shown that an equilibrium of the original game 
is an equilibrium of the game with consistent conjectures. In 
this section, the reverse relation of the equilibrium is being 


considered, i.e., an equilibrium of the DTDG with consistent 
conjectures is related to e-Nash equilibrium of the original 
DTDG. The definition of an e-Nash equilibrium of the original 
game is given as follows. 

Definition 5.3: Consider a DTDG in state conjecture for¬ 
mulation as defined in Section HIJ For e > 0, a point 
{M®*,x®*;i G Af} G {U X X) is an e-Nash equilibrium of 
the original game if Vi G Af, given u~" G U~'‘ and xi, 

J*(u**,x**;it“®*,xi) < inf X*;xi) -f e, 

where the inf is over ({t*,x*) G fli{u~";xi). 

In this section, we first analyse the case of linear-quadratic 
DTDGs with a convex cost functions and then we extend the 
analysis for a general class of DTDGs. 

1) e-Nash equilibrium for linear-quadratic DTDGs : The 
aim of this section is to relate a particular equilibrium of 
the linear-quadratic DTDG with consistent conjectures to an 
e-Nash equilibrium of the original game. We use a result 
on the exact penalty functions from the book ca to find 
the e-Nash equilibrium relation for linear-quadratic DTDGs. 
In order to state the result, consider a general optimisation 
problem denoted by P with equality constraints. 


p 

min f(x) 


s.t. 

hi{x) = 0,J = 1,. 

■,P, 


where / ; R" —R and : R” —)• R^, p < n are continuously 
differentiable functions. The following proposition provides 
the result that we use to derive the e-Nash equilibrium relation. 

Proposition 5.5: {Theorem 17.3 in M) Suppose X* is a 
strict local solution of the nonlinear programming problem 
P at which the first order KKT conditions are satisfied with 
Lagrange multipliers X*,i = l,...,p. Then x* is a local 
minimizer of cj){x; p) for all p > p*, where p* = max |A*| 

and (j){x;p) = f{x) 3-p 

It can be easily verified that if the problem P is a convex 
optimization problem, then the result in Proposition 15. 5 1 is also 
valid for a global solution. Consider a linear-quadratic DTDG 
as defined in Definition 14.31 with P G AfjMk G X. 
Therefore, a player’s problem in a linear-quadratic DTDG is 
a convex optimization problem. Let (u*,x*) G U x X 
an equilibrium of the linear-quadratic DTDG with consistent 
conjectures that we are interested in. For player i G Af, let 
(«,“**, x“”) G X be the actions and state conjectures 
of other players at the equilibrium of the linear-quadratic 
DTDG with consistent conjectures. Given (u“®*,x“**), the 
player i’s problem in a linear-quadratic DTDG with consistent 
conjectures is given by the following problem which is denoted 
by Li(M“**,x“”;xi). 

T i^k+iQl+Pk+i + ulRkul + u{ ul*) 

fce/c^ jeAA\{j} 

s.t. (u*,X*) G x“®*;Xi), 
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where, in this case, 

= {u\,... G Ul, 

V/c G /C, xl._^_l G Xl_^_l,Vk G /C, xl._^_l = Akx\ + h\u\+ 

,'^k&jc, 4+1 = 4+1,vj G Ar\{4,vfc G /c}. 

jeAA\{i} 


Here also it* = (mi, ..., and = (x 2 ,..., a;^+i). 
Consider the player i’s problem with an exact penalty function 
for the consistent state conjecture constraint with penalty 
parameter /r®. Let us denote this problem by x“®*; /i®) 

and is given by the following problem. 

Y l{^k+iQk+ixl+i+uiRkul+ 

u'^ ,x'^ Z \ 

fcG/C 

Y ^i^k^k+fj-" Y l4+i-4*+il) 

s.t. (u®,a;®) G xi), where, 


as the best response state trajectory conjectured by player i 
in the original game. The relation of {u*,x*) to the e-Nash 
equilibrium of the original game is stated as follows. 

Theorem 5.7: Consider a linear-quadratic DTDG with con¬ 
sistent conjectures with Q4i 0,Vi G Af,\/k G /C. Suppose 
(u*,x*) G is an equilibrium of this game. For player 
i G A/”, let 4+1 S ^k+i’k G /C be the state conjecture 
of player j G j\A\{*} at the equilibrium of this game. Let 
•^fc+i;Vj G A/"\{i},V/c G /C be as in Proposition 15.61 Let 
x4i 4 G /C be the player i’s best response state conjecture 
in the original game corresponding to tt“®* G C/“®. Then 
(u*,x*) is an e-Nash equilibrium of the original game with 

e = ma^Ci}, where Ci satisfy X] S 4l4+i^ 

®£^ ke/CjeAr\{i} 

2ei and /r® > where p®* is given by ©. 

Proof: Suppose {(x®*, u®*), i G A/”} is an open-loop Nash 
equilibrium of the linear-quadratic DTDG with consistent 
conjectures. That is for alH G A/^ and for each /r® > 


Vti{u ®*;xi) = {4 ,...,u^,x®2,...,x®^+iI4 G Ul, 

VfcG/c,4+i e^4i>VfcG/C, 
4+1 = ^kxl + blul + Y Kuk*^^k G 1C}. 

j€X\{i} 

Thus, we have two problems to represent the player i’s prob¬ 
lem in a linear-quadratic DTDG with consistent conjectures. 
One is Li(u“®*, x“®*; xi) with the consistent conjecture con¬ 
dition in the constraint set and another is Li(M“®*,x“®*;/i®) 
with exact penalty function for the consistent conjecture 
condition. Though the problems are different, the equivalence 
of equilibria of games having these problems can be derived 
directly from Proposition 15.51 and is stated as follows. 

Proposition 5.6: Consider a linear-quadratic DTDG with 
consistent conjectures with Q4i ^ 0,Vi G N',\/k G 1C. 
Suppose (m*,x*) € T is an open-loop Nash equilibrium of 
the game. For i G A/", let (m“®*, x“®*) G 17“® x X”® be the ac¬ 
tions and state conjectures of other players at this equilibrium. 
Let the player i’s problem in a linear-quadratic DTDG with 
consistent conjecture be given by Li(M“®*, x“®*; xi). Suppose 
the Lagrange multipliers at the equilibrium of the game 
corresponding to the consistent state conjecture constraints are 
given by A^r'+^jVj G A/’\{z},Vfc G 1C. Then (u*,x*) is also 
an equilibrium of the game with the player i’s problem given 
by Li(M“®*,x“®*;4) for > p"’ where, 

I- 

kGiC jeAf\{i} 

Proof: Since Q4i ^ G N,^k G /C, the problem 
Li(u“®*,x“®*;xi) is a convex optimisation problem which 
has a global solution. By Proposition 15.51 the result follows. 

■ 

We now use Proposition 15. 6l to derive the result which relates 
the equilibrium of a linear-quadratic DTDG with consistent 
conjectures to e-Nash equilibrium of the original linear- 
quadratic DTDG. Let {u *, x*) G A" be an equilibrium of the 
linear-quadratic DTDG with consistent conjectures. Suppose 
player i € ff plays the best response in the original game 
to (u“®*,x“®*). Then in that case let us denote x4i4 G 1C 


Y ^(4+iQ4i4+i + Y ^k'Rk^k) < 

fee/C jgTV 


fce/C 


2 (^^fc+if^fe+i^fc+i +“fe-^fc4 + 




7* 

4 


7* 

'4 - 


Y 414+1G 


Let (uj,..., u^, X 2 ,..., xj^+;^) G Hi(u“®*;xi) be the best 
response of the original linear-quadratic DTDG corresponding 
to u“®*, the actions of other players at the equilibrium of the 
game with consistent conjectures. Hence, 


Yl 2(4+1^1+14+1 + Yj 4 4 ) - 

fce/C J67V 

Y. 2 (4+iQfc+i4+i + Ufcf^fcUfc + Y. 4 ^ 


keic 


Y 


k “fe 


ieN\{i} 

414+1 -4+11)- 


By rearranging, the above inequality can be written as. 


Z! ^(4+i(34i4+i + Z Y < 

keK J67V 




l+i(3t+ixl+i+uL/i?ui 


fee/C 


- Z ' 

j6Ar\{i} 


3*\ 

k ) 


+ 


\Y Y 414+1-4+11- 

kelCjGJ\f\{i} 


Since 4l4+i — 4+il E 2ej, Vi G Af and 

fce/C jeAA\{/} 

(uj,..., u^, X 2 ,..., x^+j^) is the best response of the origi- 
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nal game, 


fee/c 


2 (^fc+iQfe+i" 


•k+l 


jeAA 


'-k^k I ^ 


inf 

(u'^ ,x‘^)GO,i(u~ 


',xi) ^ 2 

kGK 


iQ 


fc+i^fe+i^fc+i 


^k ' 


+ ^ «i''/i«<)+a 

je7VA\{i} 


Hence, {(a;®*, m®*); i G A/”} is an e- Nash equilibrium of linear- 
quadratic DTDG with e = maxjei, i € AT}. ■ 

Thus, under conditions specified in Theorem l5.7l an equilib¬ 
rium of a linear-quadratic DTDG with consistent conjectures is 
an e-Nash equilibrium of the original linear-quadratic DTDG 
for a certain e. Note that the convexity of the cost function 
and constraint set is only used in Theorem 15.71 Hence the 
result can be extended to any convex problems, not limited to 
linear-quadratic games. However, for a general case convexity 
need not hold. Therefore the next section provides an e-Nash 
equilibrium relation for a general case, but with additional 
assumptions. 

2 ) e-Nash equilibrium for general DTDGs: For a general 
class of games, we use a result on the exact penalty functions 
from the paper El by Di Pillo and Grippo for deriving 
the relation of an equilibrium of the game with consistent 
conjectures to the e-Nash equilibrium of the original game. 
The Mangasarian-Fromowitz constraint qualification (MFCQ) 
(See Definition 15.5b needs to be satisfied for the result to 
hold. For that, we consider an equivalent game having a minor 
modification from the DTDG with consistent conjectures and 
we will refer to it as alternate consistent conjecture DTDG. In 
this alternate consistent conjecture DTDG we define an exact 
penalty function game having an exact penalty function for the 
consistent conjecture condition. We derive the relation of these 
two games, the alternate consistent conjecture DTDG and the 
exact penalty function game using the result from El . The 
e-Nash equilibrium condition is provided utilising the relation 
of the exact penalty function game in turn to the DTDG with 
consistent conjectures. 

In order to make the explanation clear, let us denote the 
original game, the DTDG with consistent conjectures and the 
alternate consistent conjecture DTDG by Qo, Gc and Qa respec¬ 
tively. The difference of Ga, the alternate consistent conjecture 
DTDG and Gc^ the DTDG with consistent conjecture is that 
a player’s state conjecture is constrained to be consistent with 
the state conjecture of one other player in the game (in this 
case, the next labelled player), not all players as in Gc- 

Let f G AA be a player in a DTDG. Given G x~^ G 
AT"® and Xi, the player I’s problem in Ga is denoted by 
Pi{u~'^,x~'^-,xi) and is given by the following. 


Pfu ®,x ®;xi) 

min J®(u®,x®;zz ®,xi) 

s.t. 

,X'^ 

(u®,x®) G Oi(M“®,x“®; xi), 


where, 

n^{u~\x~'‘-,xi) = {u\,...,u"ji,p 2 ,---,XK-ei\'^k e Ul, 

Vfc G IC,xl+, G G V), 

ife+i = G /C\{l},i^+1 =4+1^^^ € /C}, 

where the next labelled player j(i) G A/" is given by 

ff + l, i€N\{N} 

j(*) = 5 (6) 

i 1, i = N. 

Using the player I’s problem given by cci), the 

open-loop Nash equilibrium in Ga can be defined as follows. 

Definition 5.4: An W-tuple of strategies {(u®*,a:®*) G C/® x 
Ar®;f G AA} is said to be an open-loop Nash equilibrium of 
Ga if Vi G A/", given (u“®*,a:“®*) G (7“® x X“®, the actions 
and state conjectures of other players at equilibrium of Ga, 

r{u^*,P*;u-^*,xi) < P{u\P;u-^*,xi), 

V('u®,i®) G rii{u~^*,x~'‘*-,xi). 

The equivalence of Gc and Ga in terms of the equilibrium set 
is given in the following proposition. 

Proposition 5.8: Consider a DTDG in state conjecture for¬ 
mulation. Suppose {(u®*,a;®*) G x X^-,i G A/"} is an 
open-loop Nash equilibrium of Gc, the DTDG with consistent 
conjectures, then {(u®*,^®*) G (7® x X®;i G A/"} is also an 
equilibrium of Ga, alternate consistent conjecture DTDG and 
vice versa. 

Proof: “ ^ ” Let {(u®*,x®*) G U® x X®; f G Af} be 
an open-loop Nash equilibrium of Gc- Then from Definition 
O at an equilibrium {(u®*,x®*),i G A/”} of Gc, since 
all players conjectures consistently, the consistency condi¬ 
tion with At — 1 players in the set Hj(M“®*,a;“®*;xi) can 
be replaced with a single player condition. Hence, at the 
equilibrium {(u®*,x®*),i G A/"} of Gc, ,x~'^*\Xi) = 

Hi(u“®*, a;“®*;xi). Thus, {(u®*,a;®*),f G A/"} is also an 
equilibrium of Ga- 

“ <J= ” Suppose {(u®*,x®®) G (7® X X®;i G A/"} is an 
equilibrium of Ga- Then from Definition 15.41 at an equilib¬ 
rium {(u®*, X®*);f G A/"} of Ga, for all z G A/", = 

G A/’\{z},Vfc G 1C. Hence, at the equilibrium 
{(m®*,x®*),z G Af} of Ga, the consistent condition with the 
adjacent player in x“®*; xi) can be replaced with 

the consistent condition for all rivals, i.e. at the equilibrium 
{(M®^,x®^);f G AC}, Hi(u-®®',x-®^;xi) = H'(u-®^, x"®®'; xi) 
and hence {(u®*, x®*), i G Af} is also an equilibrium of Gc- ■ 

Thus, by Proposition 15.81 the set of equilibria of both 
games, Gc and Ga are equivalent. Now consider a particular 
equilibrium {(u®*,x®*),z G A/”} of Gc- Our aim is to relate 
this equilibrium to an e-Nash equilibrium of Go, the original 
game. Let (m“®*,x“®*) G (7“® x X“® be the actions and state 
conjectures of the rivals of player z G A7 at the equilibrium 
of Gc- Given u“®*, the best response set of player z in Go is 
denoted by TZi{u~^*) and is given by the following. 

7^,(zz-®*) = {(zz®,x®)| 

(zz®,x®)Garg min J®(zi®, x®; u“®*, Xi)}. 
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Let 'R.i{u~'^*,x~^*) denote the best response of player i 
in Qa corresponding to which is given by the 

following. 

(u*, x*) € arg min x*; Xi)}. 

In order to make the structure of the problem same as 
the one used by Di Pillo and Grippo in ifTTll . we assume 
TZi{u~") is bounded and we introduce a compact set Vi = 
Vi{u~^*),i G JV (with non-empty interior) such that the best 
response set TZi{u~^*) C Vi. The following proposition shows 
that the best response set of alternate consistent conjecture 
DTDG iZi{u~'^*,x~'^*) is also contained in the compact set 

A. 

Proposition 5.9: Consider an equilibrium G 

f/® xX^; i G A/^} of Qc, the DTDG with consistent conjectures. 
Let G C/“® X be the actions and state 

conjectures of rivals of player i at the equilibrium of Qc. 
Let TZi{u~'‘*) and 1Zi{u~'‘*,x~^*) denote the set of best 
responses of player i in the original game Qo and the alternate 
consistent conjecture game respectively corresponding to 
x“®*). If the best response set TZi{u~‘^*) is contained in 
the compact set Vi, then the best response set x“®*) 

is also contained in Vi. 

Proof: Since there is one additional constraint of consis¬ 
tency of conjectured state in a player’s problem in Qa than 
in Qo, A C Q,i and hence ,x~") C TZi{u~'^*). 

Therefore, 1Zi{u~", x~^*) is contained in Vi. ■ 

Let (u*,x*) be an equilibrium of Qa. Given G 

[/“* X consider a game with the following problem for 
player i denoted by Pi(u~'‘*, x~''*; xi). 

Pi{u~",x~^*-,xi) min J®('u®, x®; u“®*, xi) 

U'^ 

s.t. (^“jX®) G rii(M“®*,x“®*; xi) nDi. 

Note that the feasible set in this game is an intersection of 
the interior of the compact set Vi and the feasible set of 
Pi{u~^*,x~";xi). Since Vi contains the best response of 
player i in Qa, the solution of problem x“®*; xi) is 

also a solution of PA^”®*,x“®*;xi). Therefore, an equilib¬ 
rium of Qa is also an equilibrium of the game with player 
Ls problem given by Pi(it“®*,x“®*;xi). Now consider the 
player Ls problem given by Pi(u“®*, x“®*; Xi) with an exact 
penalty function for the consistent state conjecture constraint 
with penalty parameter pi. Let us denote this problem by 
Qi{u~'^*, x~"; Pi) which is given by the following. 

min J®(u®, X®; u“®*, xi) -I- /rj|x® — x-’^®)*] 
s.t. (u®, X®) G ni(M“®*; xi) n Pj, 

where > 0 is the penalty parameter and 
is the state conjecture of the next labelled player at the 
equilibrium of Qc, where j{i) given by (|6]l. We will refer to 
the game with player z’s problem given by Qi{u~^* ,x~^*; pf) 
as exact penalty function game. The relation of equilibria of 
Qa and exact penalty function game can be deduced directly 
from a result in the paper IfTTll which is given in Proposition 


I5.10I For the result to hold, the Mangasarian-Fromowitz 
constraint qualification (MFCQ) is required to satisfy. For 
defining MFCQ, consider the equality constrained optimisation 
problem given by P. 

Definition 5.5: The MFCQ holds at x G M" for problem 
P if \/hj{x),j — 1,... ,p are linearly independent and there 
exist a 2 G M" such that Xhj{x)'z = 0, j = 1,... ,p. 

By the theorem of alternatives, the MFCQ can be restated as 
follows. The MFCQ holds at x G M®® if there exist no Vj,j = 

p 

l,...,p such that = 0, {vj,j = 1, • ■ • ,p) pO- 

i=i 

Note that MFCQ is satisfied at the equilibrium of Qa since the 
gradients of the consistency constraints at equilibrium of Qa 
are linearly independent. In the following proposition we state 
the result which is derived from ifTTl . 

Proposition 5.10: {Theorem 4 in 1171 1 Let (u*,x*) G P 
be an equilibrium of Qa. Assume Mangasarian-Fromowitz 
constraint qualification (MFCQ) is satisfied at (u*,x*) and 
suppose TZi{u~") C Vi. Then there exist a G Af such 
that for all pi G [^*,oo), (u*,x*) is an equilibrium of the 
exact penalty function game. 

It can be seen that though in the paper by Di Pillo and 
Grippo, all the equality constraints are penalized, the result still 
holds even if we penalize a subset of the equality constraints 
as in our case. Let (u*,x*) G T be an equilibrium of 
DTDG with consistent conjectures. Suppose for (m“®*,x“®*), 
the player i plays the best response in the original game. In 
that case as we have done in the linear-quadratic case, let us 
denote x® as the best response state trajectory conjectured by 
player i in the original game. The relation of an equilibrium 
in Qc to the e-Nash equilibrium of Qo is stated as follows. 

Theorem 5.11: Consider a DTDG with consistent conjec¬ 
tures given by Qc. Suppose {u*,x*) G T is an equilibrium of 
Qc. Suppose xlO)* g 

is the state trajectory conjectured 

by player j{i) G AC at the equilibrium of Qc, where is 

given by (|6l). Suppose that lZi{u~^*) C Vi, for all i G Af. 

Let (u®,x®) G lZi{u~^*) be a best response state trajectory 

conjectured by player i in Qo corresponding to (m“®*, x“®*). 

Then (m*,x*) is an e-Nash equilibrium of the original game 

with e = maxjei}, where e^ satisfy p,i|x® — x-^T®)*! < and 
i^J\f 

Pi G [p*,oo), where p* given by Proposition 15.101 

Proof: Suppose {(u®*,x®*),i G AC} G P is an open- 
loop Nash equilibrium of Qc. Since the equilibrium set of 
Qc is equivalent to Qa, {(u®*,x®*) G W x X^,i G AC} is 
also an equilibrium of Qa. Since MFCQ is satisfied at an 
equilibrium of Qa, by Proposition 15.101 the equilibrium set 
of Qa is equivalent to that of the exact penalty function game 
for all Pi G [pi,oo). Hence, for all i G AC and for some 
p^ G [p*,oo). 


r{p* 


X®*; u ®*, Xi) < 


J®(u®,x®;rt ®*,xi)-|- 
,V(m®,xQ GOi(u-®*;xi)n25,. 


Let (u®,x®) G Hi(it“®*;xi) be a best response of player 
i in Qo corresponding to the equilibrium actions u“®* of Qc. 
Since the best response set lZi{u~'^*) C Vi, we can write 
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(u*,x®) e Vi- Since (u*,x*) e VLi{u~'^*\xi) CiVi, 

X**;m“**, xi) < J*(u*,X®; m“**, xi) + /x^jx® — 

Since (u®,x®) is a best response of the original game, 

J i / i-* i* —i-* „ \ ^ 

(u , X ; It , Xi j < 


inf 

{u'^ ,x'^)^di{u~'^* ’jXi) . 


J*(u*,x*;u **,xi) 


+M^|x*-x^«*| 


Since/ii|x®—x-^^®)*! < ei,Vt G TVand e = maxjei; t G A/”}, 


J®(u”,x®*;u-®*,xi) < inf 


J\u\x^;u ®*,xi) +ei, 


where the inf is over (m®,x®) G 17^(12“®*;xi). That is 
(u®*,x®*;t G A/") is an e-Nash equilibrium of the original 
game. ■ 

This result gives conditions for an equilibrium of the game 
with consistent conjectures to be an e-Nash equilibrium of the 
original game. A significant factor which determines the e- 
Nash equilibrium relation is the deviation of state conjectures 
at the equilibrium of Qc to the best response state conjecture 
of a player in Qo corresponding to the state conjectures at the 
equilibrium of Qc- 


VI. Conclusion 

The paper provides new results regarding the existence of 
open-loop Nash equilibria in DTDGs. A new approach for 
analysis of DTDGs using the state conjecture formulation is 
the pivotal step which leads to the new results. Using this 
formulation, a solution of an optimisation problem is related 
to an equilibrium for a certain class of DTDGs. The first result 
that we provide is for a class of games called quasi-potential 
DTDGs. Many games including linear-quadratic DTDGs under 
certain conditions comes under the class of quasi-potential 
DTDGs. Further, we modify the DTDG with an additional 
constraint of consistency of the conjectured states for which 
the game has a shared constraint structure. Although the 
game is different from the original game, an equilibrium 
of the original game is also an equilibrium of the DTDG 
with consistent conjectures. Utilising the shared constraint 
structure, conditions for the existence of equilibria for a class 
of games called potential DTDGs are provided in this modified 
game with consistent conjectures. A detailed section on the e- 
Nash equilibrium gives a relation of an equilibrium of the 
DTDG with consistent conjectures to the original game. 
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